Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dataframe columns / Filter Fhir elements by FhirPath #105

Open
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

Mandalka
Copy link
Contributor

Add Pandas dataframe columns / Filter Fhir elements by FhirPath / the Python implementation fhirpath-py (f.e. for easier filtering/analysis of codes of certain/different codesystems)

@dermatologist dermatologist added the enhancement New feature or request label Dec 30, 2022
@Mandalka
Copy link
Contributor Author

Mappings format is yet from the ETL point of view

    f.mappings_fhirpath_to_column = {
      "code.coding.where(system = 'http://snomed.info/sct').code": 'code_snomed',
      "code.coding.where(system = 'urn:mysystem').code": 'code_mysystem',
    }

Since fhir paths longer/more difficult to read than the dataframe column names choosen by users maybe it would be easier for users to overview/manage the mappings in the other direction (more from the dataframe/analysis point of view) like

    f.mappings_column_from_fhirpath = {
       'code_snomed': "code.coding.where(system = 'http://snomed.info/sct').code" ,
       'code_mysystem': "code.coding.where(system = 'urn:mysystem').code",
    }

Opinions / proposals for better property name?

@Mandalka
Copy link
Contributor Author

Mandalka commented Jan 3, 2023

If no other wishes/proposals would change name and direction to

f.columns_by_fhirpaths = {
  "mycolumnname": "myfhirpath",
  ...

}

Markus Mandalka added 4 commits January 13, 2023 12:02
@Mandalka
Copy link
Contributor Author

Mandalka commented Jan 13, 2023

  • Changed direction and name of mappings like described in comment before

  • Integrated FHIR Path extraction with FHIR search to dataframe.

Further plan:

  • Integrate FHIR Path extraction to columns with parallel processing of JSON bundles from folder (fhiry.parallel.process)

  • Add FHIR Path extraction to usage documentation

@dermatologist
Copy link
Owner

Please mark this as done once completed, so that I can merge this. Thanks!

@Mandalka
Copy link
Contributor Author

Mandalka commented Jan 30, 2023

It is done for FHIR bundle files and FHIR search, but (like documented in readme) not yet for ndjson (no time next weeks and not sure if first to add quick and dirty (by copying the code parts from fhiry.py) or better but more long term more DRY (but more effort/have to compare the differences between fhiry.py and fhirndjson.py / maybe some architecture discussion for somehow unifying or inheriting or sharing codeparts between fhiry.py and fhirndjson.py maybe by defining a property with maybe (optional) prefix "resource." for json bundles which not used or empty for import of ndjson (i yet didnt use in FHIR context/yet no experience/deeper look on fhirndjson.py)

@dermatologist
Copy link
Owner

I have released a new version with the previous PR. Thanks for the effort!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants